A storage system typically comprises one or more storage devices into which information may be entered, and from which information may be obtained, as desired. The storage system includes a storage operating system that functionally organizes the system by, inter alia, invoking storage operations in support of a storage service implemented by the system. The storage system may be implemented in accordance with a variety of storage architectures including, but not limited to, a network-attached storage environment, a storage area network and a disk assembly directly attached to a client or host computer. The storage devices are typically disk drives organized as a disk array, wherein the term “disk” commonly describes a self-contained rotating magnetic media storage device. The term disk in this context is synonymous with hard disk drive (HDD) or direct access storage device (DASD).
Storage of information on the disk array is preferably implemented as one or more storage “volumes” of physical disks, defining an overall logical arrangement of disk space. The disks within a volume are typically organized as one or more groups, wherein each group may be operated as a Redundant Array of Independent (or Inexpensive) Disks (RAID). Most RAID implementations enhance the reliability/integrity of data storage through the redundant writing of data “stripes” across a given number of physical disks in the RAID group, and the appropriate storing of redundant information (parity) with respect to the striped data. The physical disks of each RAID group may include disks configured to store striped data (i.e., data disks) and disks configured to store parity for the data (i.e., parity disks). The parity may thereafter be retrieved to enable recovery of data is lost when a disk fails. The term “RAID” and its various implementations are well-known and disclosed in A Case for Redundant Arrays of Inexpensive Disks (RAID), by D. A. Patterson, G. A. Gibson and R. H. Katz, Proceedings of the International Conference on Management of Data (SIGMOD), June 1988.
The storage operating system of the storage system may implement a high-level module, such as a file system, to logically organize the information stored on the disks as a hierarchical structure of data containers, such as directories, files and blocks. For example, each “on-disk” file may be implemented as set of data structures, i.e., disk blocks, configured to store information, such as the actual data for the file. These data blocks are organized within a volume block number (vbn) space that is maintained by the file system. The file system may also assign each data block in the file a corresponding “file offset” or file block number (fbn). The file system typically assigns sequences of fbns on a per-file basis, whereas vbns are assigned over a larger volume address space. The file system organizes the data blocks within the vbn space as a “logical volume”; each logical volume may be, although is not necessarily, associated with its own file system. The file system typically consists of a contiguous range of vbns from zero to n, for a file system of size n+1 blocks.
A known type of file system is a write-anywhere file system that does not overwrite data on disks. If a data block is retrieved (read) from disk into a memory of the storage system and “dirtied” (i.e., updated or modified) with new data, the data block is thereafter stored (written) to a new location on disk to optimize write performance. A write-anywhere file system may initially assume an optimal layout such that the data is substantially contiguously arranged on disks. The optimal disk layout results in efficient access operations, particularly for sequential read operations, directed to the disks. An example of a write-anywhere file system that is configured to operate on a storage system is the Write Anywhere File Layout (WAFL®) file system available from Network Appliance, Inc., Sunnyvale, Calif.
The storage system may be configured to operate according to a client/server model of information delivery to thereby allow many clients to access the directories, is files and blocks stored on the system. In this model, the client may comprise an application, such as a database application, executing on a computer that “connects” to the storage system over a computer network, such as a point-to-point link, shared local area network, wide area network or virtual private network implemented over a public network, such as the Internet. Each client may request the services of the file system by issuing file system protocol messages (in the form of packets) to the storage system over the network. By supporting a plurality of file system protocols, such as the conventional Common Internet File System (CIFS) and the Network File System (NFS) protocols, the utility of the storage system is enhanced.
Certain known examples of file systems are capable of generating a snapshot of the file system or a portion thereof. Snapshots and the snapshotting procedure are further described in U.S. Pat. No. 5,819,292 entitled METHOD FOR MAINTAINING CONSISTENT STATES OF A FILE SYSTEM AND FOR CREATING USER-ACCESSIBLE READ-ONLY COPIES OF A FILE SYSTEM by David Hitz et al, issued Oct. 6, 1998, which is hereby incorporated by reference as though fully set forth herein. “Snapshot” is a trademark of Network Appliance, Inc. It is used for purposes of this patent to designate a persistent consistency point (CP) image (PCPI). A PCPI is a read-only, point-in-time representation of the storage system, and more particularly, of the active file system, stored on a storage device (e.g., on disk) or in other persistent memory and having a name or other unique identifier that distinguishes it from other PCPIs taken at other points in time. A PCPI can also include other information (metadata) about the active file system at the particular point in time for which the image is taken. The terms “PCPI” and “snapshot” shall be used interchangeably through out this patent without derogation of Network Appliance's trademark rights.
PCPIs can be utilized as a form of backups for an active file system. To provide for improved data retrieval and restoration, PCPIs should be copied to another file system different than the volume or file system on which the PCPI was generated. In one known example, a backup storage system is utilized to store PCPIs and manage a collection of PCPIs according to a user defined set of options. Backup storage systems are described is in further detail in U.S. Pat. No. 7,475,098, issued on Jan. 6, 2009, entitled SYSTEM AND METHOD FOR MANAGING A PLURALITY OF SNAPSHOTS, by Hugo Patterson et al., which is hereby incorporated by reference.
In environments utilizing PCPIs as a form of backup, oftentimes the same data may be stored in multiple locations on the backup storage system. This may occur when, for example, a single data container contains a plurality of sections that contain the same data or when multiple data containers within a volume or other collections of data (i.e., data collection) contain the same data. Known techniques for reducing the amount of duplicate data may be utilized in a storage system to reduce the number of duplicate data blocks in, e.g., the active file system. However, in backup storage system environments where PCPIs are utilized, long term retention of the PCPIs may limit the effectiveness of conventional data deduplication techniques, primarily because the benefit of a data deduplication procedure, i.e., a reduction in the number of duplicate data blocks being stored, typically requires modifications to various data pointers within the data collection. As a PCPI is read only, the blocks in the PCPI remain “locked” (and thus not subject to deduplication) until the PCPI is deleted. In a non-backup storage system, PCPIs are typically deleted after a few days, so the long term PCPI retention limitation is minimized.